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Dear Sir/Madam: 

This Declaration is submitted to establish invention of the. subject matter of the 
present patent application prior to the publication date of February 12, 2004 of U.S. Patent 
Application No. US 2004/0027363 (hereinafter referred to as "Allen"). 

1 . The person making this Declaration is inventor David C. Collins. 

2. Accompanying this Declaration are Exhibit A and Exhibit B to establish reduction 
to practice of the subject matter of the present patent application in the United 
States prior to February 12, 2004. 

3. Exhibit A (10 pages) includes a Hewlett-Packard Company (HP) Invention 
Disclosure and accompanying attachment (hereinafter referred to collectively as 
the "Disclosure"). 

4. Exhibit B is a page from a Laboratory Notebook of inventor David C. Collins 
(hereinafter referred to as the "Laboratory Notebook Page"). 

5. The Disclosure and the Laboratory Notebook Page were prepared by the inventor 
in the United States prior to February 12, 2004. 

6. The Disclosure and the Laboratory Notebook Page were witnessed in the United 
States prior to February 12, 2004. 

7. The Disclosure was submitted by the inventor to the HP Legal Department in the 
United States prior to February 12, 2004. 

8. The Disclosure was assigned HP Patent Disclosure No. 200400519. 
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9. The Disclosure and the Laboratory Notebook Page describe the subject matter of 
the present patent application - i.e., the subject matter of independent claims 1, 
1 1, 20, and 25 - as follows. 



Claims 


Disclosure & Laboratory Notebook Page 


Claim 1: 


The Disclosure (pp. 1-10) describes the 


<C A tnpfTinrl rvF Hi ci'nl a \nn ex o-n i'iti p ctf» "1X71+11 q 

xTl. HJLC'l/lUJU. Ul U-loJJJLCtyill^ CLLL lllldgC WILLI ca- 


generation of sub-frames for display at 


di sr> lav devi ce the method nomnri^iTiP"" 


spatially offset positions to form a displayed 




image. 


Claim 1 1 : 






Wobulation, as referenced on pp. 1, 2, and 4 


"A system for displaying an image, the 


of the Disclosure, refers to the display of sub- 


system comprising. 


frames at spatially offset positions to display 


Claim 20: 


an image using a display device. 


"A system for generating sub-frames for 


The Laboratory Notebook Page shows 


display at spatially offset positions to 


pictures labeled "Center Adaptive w/ 


generate the appearance of an image, the 


History" and "Adaptive w/ History" that each 


system comprising:" 


represent a displayed image. 


Claim 25: 




"A computer-readable medium storing 




computer-executable instructions, which, 




when executed by a computer processing 




system, cause the system to perform a 




method of generating a sub-frame image 




which comprises a plurality of sub-frames for 




display at spatially offset positions to 




generate the appearance of an image, 




comprising:" 




Claim 1 : 


Image data is shown on p. 6 of the Disclosure 


"receiving image data for the image;" 


(a region of interest "ROI" for a given sub- 




frame pixel value is shown relative to the 


Claim 11: 
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"a buffer adapted to receive image data for 


image data). 


the image;" 


Image data is operated on by the set of 




computer- ex ecutaoie instructions listed on 


"means for receiving image data 


pp. 8-10 of the Disclosure. 


corresponding to the image;" 


The Laboratorv Notebook Pa?e slinws a 


Claim 25: 


picture labeled (Jrigmal mat represents 




received imaee data 


"receiving image data corresponding to the 


image;" 




Claim 1 : 


On p. 4 of the Disclosure, "[a]ll of the sub- 


"generating first and second sub-frames to 


frames are processed together at the same 


include first and second sub-frame pixel 


time, and the sub-frames are intertwined." 


values, respectively, and wherein the first 


The diagram on p. 4 of the Disclosure 


sub-frame pixel value is calculated using the 


illustrates how the sub-frames are intertwined 


image data and the second sub-frame pixel 


in one embodiment. 


value;" 


Pp. 6-7 of the Disclosure describe the ! 


Claim 11: 


generation of final sub-frame pixel values for 


"an image processing unit configured to 


the sub-frames. 


generate first and second sub-frames 


As shown in the diagrams and described on 


comprising a plurality of rows of sub-frame 


pp. 6-7 of the Disclosure, each final sub- 


pixel values, wherein each of the sub-frame 


frame pixel value is generated using values 


pixel values in each of the plurality of rows is 


from a region of interest. 


calculated using the image data and at least 


As shown in the diagrams on p. 6 and p. 7 of 


one sub-frame pixel value from a previous 


the Disclosure, the region of interest for the 


one of the plurality of rows; 


second and subsequent rows of image data 


Claim 20: 


includes final sub-frame values, i.e., "Final 


"means for generating a plurality of 


values calculated". See also p. 6 ("[m]any of 


rows of initial values using the image data; 


the final sub-frame values will already have 




their final values computed.") 
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means for accessing a row of history 
values generated using the image data; and 

means for generating a sub-frame 
pixel value using the row of history values 
and the plurality of rows of initial values." 

Claim 25: 

"generating a first plurality of initial 
values associated with a first pixel which 
corresponds to a first one of the plurality of 
sub-frames using the image data; 

generating a first sub-frame pixel 
value using the image data and the first 
plurality of initial values, wherein the first 
sub-frame pixel value comprises a first 
history value; 

generating a second plurality of initial 
values associated with a second pixel which 
corresponds to a second one of the plurality 
of sub-frames using the image data; and 

generating a second sub-frame pixel 
value using the image data, the second 
plurality of initial values, and the first history 
value." 



As shown in the diagrams on p. 6 and p. 7 of 
the Disclosure, final sub-frame pixel values 
for the second and subsequent rows in the 
diagram on p. 6 are generated using final sub- 
frame pixel values from the previous row in 
the region of interest as shown in the diagram 
on p. 7. 

The final sub-frame pixel values (e.g., the 
previous row) used to generate a given final ) 
sub-frame pixel value are also referred to as 
history values. See Disclosure diagram on p. 
8 ("Historic Values"). 

The "Algorithm Steps" on p. 8 of the 
Disclosure show the generation of rows of 
initial values. 

The set of computer-executable instructions 
listed on pp. 8-10 of the Disclosure performs 
a method of generating a final sub-frame 
pixel value (guess 1_0M) using other final 
sub-frame pixel values (e.g., final0_5M, 
fmalO_4M, fmalO_3M, fmal0_2M, 
final0_lM, and fmal0_0M,) and initial values 
(e.g., image 1_1M to imagel_5M, 
image2_lM to image2_5M, image3_lM to 
image3_5M, image4_lM to image4_5M). 

The "Algorithm Steps" on the Laboratory 
Notebook Page show the generation of rows 
of initial values. 



Claim 1: 

"alternating between displaying the first sub- 



Wobulation, as referenced on pp. 1,2, and 4 
of the Disclosure, refers to the display of sub- 
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frame, including displaying the first sub- 


frames at spatially offset positions using a 


frame pixel value, in a first position and 


display device. 


displaying the second sub-frame, including 




displaying the second sub-frame pixel value, 




in a second position spatially offset from the 




first position." 




Claim 11: 


< 


"a display device adapted to alternately 




display the first sub-frame in a first position 




and the second sub-frame in a second 




position spatially offset from the first 




position." 





10. The subject matter of the present patent application was tested as follows. 

a. The set of computer-executable instructions is listed on pp. 8-10 of the 
Disclosure embodies the method of generating a final sub-frame pixel 
value described in the Disclosure. 

b. The set of computer-executable instructions listed on pp. 8-10 of the 
Disclosure was written prior to February 12, 2004. 

c. The set of computer-executable instructions listed on pp. 8-10 of the 
Disclosure was executed with at least one test image on at least one 
computer system to generate final sub-frame pixel values for sub-frames 
for the test image prior to February 12, 2004. 

d. The sub-frames generated for the test image were displayed on at least one 
display device to confirm the successful generation of sub-frames and the 
successful reproduction of the test image prior to February 12, 2004. 
Laboratory Notebook Page (pictures labeled "Center Adaptive w/ History" 
and "Adaptive w/ History"). 
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1 1, The e&ecu^^ instructions listed on pp. 8-10 of 
the Disclosure oa the computer system to generate the final sub-frame pixel values 
for the sub-frames combined with the display of the sub-frames on the display 
device demonstrates actual working conditions or a realistic simulatiQii of working 
conditions for the subject matter of the present patent application, 

12. The successful generation of sub-frames and successful reproduction of the test 
image demonstrates utility beyond a probability of Inilrire fo subject matter of 
the present patent application, 

13. The test results -tot- dtest^ of the present pa^ 

successful generation of sub-fraines and successful reproduction o f the test image, 
are reproducibie by executing the set of computer-executable instructions listed on 
pp. 8- ID of the Disclosure on a suitable computer systemx wi^ 
image to generate final sub-ixame for the image and 

displaying, the ^ub-ffames a suitable display device to reproduce the image; 

14, As demonstrated by this Declaration, Exhibit A 9 anc^ Exhibit B; the subject matter 
of the present patent application was reduced to practice in the United States prior 
to the publicalipn date of February 12, 2004 of Allen. 

As a person signing below, I hereby declare that all statements made herein c^Mry 
om^ mad&oninfcn^ 
to be true; and further that these statements were made^ w 

statements and the like so made are punishable by fine or imprisonment, or both, un 
Section 1001 of Title 1 8 of the United States Code, npd thatspeb w 
jeopardize the validity of the application or any patent issued thereon. 





David C. Collins 
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200400519: A Practical ImplementatL. 



Innovation Number 
200400519 



Disclosure Summary 



ISSIIII:::; 




Title A Practical Implementation of Multipass Adaptive Wobulation 

Abstract Optimal subframe generation for wobulation can be implemented with an interative algorithm. 
This disclosure describes a practical method for implementing multiple passes of the 
wobulation subframe generation algorithm. 

Attachments j| AdaptiveJKemeLWithJistory2.doc [188928 bytes] 
Inventors David C Collins 



Invention Disclosures 




invent 



Disclosure No. 200400519 



■■ ,.„„,,•:,,,, 



PD No. 
200400519 



Date/Time Submitted 




The information contained in this document is HP CONFIDENTIAL and may not be disclosed to others without prior authorization. 
Submit this disclosure to the HP Legal Department as soon as possible. No patent protection is possible until a patent application 
is authorized, prepared, and submitted to the Government. 


QtZp General Information 


Title 


A Practical Implementation of Multipass Adaptive Wobulation 


Abstract 


Optimal subframe generation for wobulation can be implemented with 
an interative algorithm. This disclosure describes a practical method 
for implementing multiple passes of the wobulation subframe 
generation algorithm. 


Projects 




Products 




|H Attachments 

* 


Attachments 





Descripti o n of I n vention 



Problems Solved 



Optimal subframe generation for wobulation is computationally 
complex. Niranjan at HP Labs has previously disclosed an iterative 
method for solving the optimal subframe generation problem. This 
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has been called the adaptive wobulation method. Running multiple 
iterations of the adaptive algorithm is expensive in terms of memory 
bandwidth, and is not practical for a consumer grade product. This 
disclosure describes a method for computing multiple passes of the 
adaptive algorithm with limited memory bandwidth requirements. 




Prior Solutions 


Two straight forward methods exist for optimal subframe generation 
using the adaptive method. The first method is to simply run the 
algorithm for multiple iterations. This results in the following steps: 1) 
Create the initial subframe 2) Generate a simulated image (i.e. merge 
the subframes together) 3) Calculate the error by subracting the 
simulated image from the original image 4) Feedback the error to 
update the subframes goto step 2 This method requires a great deal 
of memory bandwidth as the simulated image must be stored in 
memory, and the subframes must be stored in memory for each 
iteration. The second method is to apply the iterative algorithm on a 
portion of the image, and then calculate the final subframe value in a 
single step. Once a value have been calculated, the region of interest 
is shifted and a new value is calculated. This method requires a small 
amount of memory bandwidth, but many calculations are redundant, 
ana tne internal memory requirements ot me calculation naraware are 
still quite large. The region of interest grows as the number of 
interations grows. Thus, to date, only one pass of the adaptive 
algorithm has been seriously proposed. 




Description 


The description is given in the attached document. In summary, the 
invention reduces the number of rows of data that are required for the 
multipass adaptive wobulation algorithm. This is accomplished by 
making a small compromise for some of the initial values, and by 
keeping track of the previous row of final subframe values. 




Advantages 


This invention enables multipass wobulation to be implemented in an 
aaoio witnout increasing tne memory Danawitn requirements, miso, 
the amount of on chip memory is kept relatively small. The techniques 
in this invention can be applied in a straight forward fashion for the 
simplified center adaptive algorithm that was recently disclosed. 




Invention History 






Published 






Announced 
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1 1 




Related Disclosure 
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Adaptive Kernel With History 

The following explanation is assuming 4 position wobulation. Thus, four low resolution 
subframes must be generated subframes — one for each image position. All of the 
subframes are processed together at the same time, and the subframes are intertwined. 
Thus, every four pixel will correspond to a different subframe. The following diagram 
illustrates the idea. 




To compute the adaptive algorithm, subframes are computed, a simulated image ( the 
four subframes merged together) is computed, an error image is computed. The error is 
averaged, and then feedback into the subframes. This process is repeated iteratively, and 
within a few iterations, this method converges to the solution. The standard adaptive 
algorithm is briefly defined below: 



Simulation Kernel 



4 4 



J_ 

4 



0 

0 0 0 



Error Kernel 
0 



0 



i 

4 

4 4 




Origina 
I 

Image 



Iterative Algorithm 
Simulation = K s * Subframes 
Error = Image - Simulation 
Error avg = K e * Error 
Subframes = Subframes + ax Erroravg 



In order to calculate the subframe value for the a single pixel a 4x4 ROI is required for 
the one iteration. The diagram below illustrates the required pixel values. 
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ROT for Simulation 



For two iterations, a 6x6 ROI is required. 



Pixel Of 
Interest 



ROT for Error Averaging 



ROT for Tnitial Values 



ROI for 2 
Iterations 



For 3 iterations an ROI of 8x8 is needed, and in general for n iterations an ROI of 
(2n+2)x(2n+2) is required. 

Consider 3 iterations of the adaptive algorithm. To compute the optimal adaptive 
algorithm, an 8x8 ROI is required. 

















m 



















































































































ROI for 3 
Iterations 



Gray Pixels Only 
Required for Initial Value 



The bottom row of the ROI is only required to generate the initial guess for the pixel 
values that are at least 2 pixels away from the primary pixel of interest. There is very 
little impact in eliminating the last row from the ROI. This just implies that for some of 
the pixels a simplified calculation will have to be used for the initial subframe value. 
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To generate the subframe values, the region of interest is positioned such that the pixel of 
interest is located over each pixel location. Typically, the ROI starts in the upper left 
corner, and the image is processed in a raster format. Thus, the first row of subframe 
values are calculated, followed by the second, and then the third row of data. The 
following image illustrates how the ROI is moved across the image and the subframe 
values are generated. 




Since the image processing is typically done in a raster pattern. Many of the final 
subframe values will already have their final values computed. The diagram below 
illustrates the pixel locations with final values. 




Final Values calculated 



ROI for 3 
Iterations 



Gray Pixels Only 
Required for Initial Value 



Bottom row ignored 
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Now if we look at the calculations we can see that instead of using three pixel rows of 
image data above the pixel of interest, we can substitute the final calculated values 
instead. Thus, we can compute the 3 -pass adaptive algorithm by using 1 row of the 
previously calculated subframe values along with 4 rows of the original image. This 
results in the following ROI: 















A 






































































Final values calculated 



ROI for 3 Iterations with 
History (Final Values) 



Gray Pixels Only 
Required for Initial Value 



Bottom row ignored 



This same approach can be used for a variety of iterative or adaptive algorithms. 
Essentially two main simplifications have been done. First, instead of using a number of 
image rows above the pixel of interest, only one row is used, and this row contains the 
calculated subframe values and not the original image values. The second simplification 
comes from truncating the kernel in heighth, but not in width. It should also be obvious 
that these techniques can be applied to achieve more than 3 iterations. Three iterations 
was used for the example because it highlights the novelties of the invention, and three is 
about the right number of iterations to achieve almost all of the benefits of the iterative 
algorithm. 



The benefits of this algorithm are best appreciated by an ASIC implementation. With 
this method the ASIC could be designed to include enough onboard memory to store the 
original image rows and one row of history (calculated subframe values). The image 
would be processed as it enters the ASIC. Only the final subframe values would need to 
be stored in external memory, and then read out at the proper time. Thus the memory 
bandwidth only has to support the writing and reading of a full frame of data. Also, as 
the kernel grows in height, more memory would be required on the ASIC. This explains 
why the goal was to reduce the number of rows needed by the ROL 
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In summary, the 3-pass adaptive kernel with history can be implemented by a relatively 
simply algorithm. The following steps are performed once per pixel location. 



Algorithm Steps 

1 Calculate Adaptive C4 (Initial 
Guess) 

2 Calculate Sim C4 (Convolve with 
Ks) 

3 Calculate Error C4 (Convolve with 
Ke) x 

4 Calculate Sim C3 

5 Calculate Error C3 " 

6 Calculate Adaptive C3 (x = x + a 
Error ) 

7 Calculate Sim C2 

8 Calculate Error C2 

9 Calculate Adaptive C2 

10 Calculate Sim Ci 

1 1 Calculate Error Ci 



Historic 
Values 



Pixel of 
Interest 



c 


c 

1 ......... 


C 

mArnrn 


C 


C 


C 




















f 












1 









Adaptive 
Values 



Note: 

C5 is Column 5 
C4 is Column 4 
Etc... 



And for the truly inspired, the 3 pass adaptive algorithm can be implement with the 
following C++ code. The same approach has also been applied to the simplified center 
adaptive algorithm: 

unsigned char AdaptiveStandardKernel :: Calculate 
( unsigned char finalO, 

unsigned char imagel, 

unsigned char image2, 

unsigned char image3, 

unsigned char image4 

) 
{ 

int temp; 

int x,y; // temporary values to explicitly reduce the number of calculations 

shiftValues () ; 



finalO_5M' = finalO; 
image 1_5M = imagel; 
image 2_5M = image 2; 
image 3_5M = image 3; 
image4_5M = imaged; 



// calculate guess for column 4 -> average of 4 image pixels 



x = imagel_4M+imagel_5M; 

y = image 2_4M+ image 2_5M; 

guessl_4M = (x+y)»2; 

x = image3_4M+image3_5M; 

guess2_4M = (x+y)»2; 

y = image4_4M+image4_5M; 

guess3_4M « (x+y)>>2; 

guess4_4M = y»l; 

// compute sim column 4 
x = finalO_3M+finalO_4M; 

y = guessl_3M+guessl_4M; 

int siml_4 = x+y; 

x = guess2_3M+guess2_4M; 

int sim2_4 = x+y; 
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y = guess3_3M+guess3_4M; 

int sim3_4 = x+y; 

x = guess4_3M+guess4_4M; 

int sim4_4 = x+y; 



int 


errl_4 


= (imagel 4M<<2) 




4 ; 


/ / pnl nmn 4 

/ / L- _L LJ.1 111 1 ri 


int 


err2_4 


= (image2 4M«2) 


— s im2 


4 ; 




int 


err3_4 


= (image3~4M«2) 


- sim3_ 


~4; 




int 


err4_4 


= ( image 4_4M«2) 


- sim4_ 


_4/ 




// compute 


sim column 3 








X 




= finalO 2M+finalO 


3M; 






y 




= guessl_2M+guessl 


~3M; 






int 


siml_3 


= x+y; // column 3 








X 




= guess2_2M+guess2 


_3M; 






int 


sim2_3 


= x+y; 








y 




= guess3_2M+guess3 


_3M; 






int 


sim3_3 


= x+y; 








X 




— gues s4_2M+guess4 


_3M; 






int 


sim4 3 


= x+y; 








int 


errl_3 


= (imagel_3M«2) 


- siml_ 


3; 


// column 3 


int 


err2_3 


= (image2_3M«2) 


- sim2_ 


~3; 




int 


err3_3 


= (image3_3M«2) 


- sim3_ 


~3; 




int 


err4_3 


= (image4_3M«2) 


- sim4_ 


"3; 





// compute guess column 3 

// divide by 4 for average error, 4 for simulation, & 4 for numerator 
x = errl_3+errl_4 ; 

y = err2_3+err2_4; 

temp = x+y; 

temp = (temp * alphalM) >> 6; 
temp = temp + guess 1_3M; 
guess 1_3M = max (0,min (temp, 255) ) ; 



x = err3_3+err3_4 ; 

temp - x+y; 

temp - (temp * alpha 1M) » 6; 
temp = temp + guess2_3M; 
guess2_3M = max ( 0 , min (temp, 255 )) ; 

y = err4_3+err4_4 ; 

temp = x+y; 

temp = (temp * alphalM) >> 6; 

temp = temp + guess3_3M; 

guess 3_3M = max ( 0 , min (temp, 255 )) ; 

temp = y; 

temp = (temp * alphalM) >> 5; 
temp - temp + guess4_3M; 
guess4_3M - max (0, min (temp, 255) ) ; 

x = finalO_lM+finalO_2M; 

y = guessl_lM+guessl_2M; 

int siml_2 = x+y; // column 2 
x = guess2_lM+guess2_2M; 

int sim2_2 = x+y; 

y = guess3_lM+guess3_2M; 

int sim3_2 » x+y; 

x = guess4_lM+guess4_2M; 

int sim4_2 = x+y; 

int errl_2 = (imagel_2M«2) - siml_2; 
int err2__2 = (image2_2M«2 ) - sim2_2; 
int err3__2 « (image3_2M«2 ) - sim3_2; 
int err4_2 = (image4_2M«2 ) - sim4_2; 

x = errl_2+errl_3; 

y = err2_2+err2_3; 

temp = x+y; 

temp = (temp * alpha2M) » 6; 



// guess 1_3 

// clip value 

// guess 2_3 

// clip value 

// guess 3_3 

// clip value 

// guess 4_3 

// by 5 since x is not available 

// clip value 



// column 2 



// guess 1_2 



) 
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temp = temp + guess 1_2M; 

guess 1_2M = max (0,min (temp, 255) ) ; 

x - err3_2+err3_3; 

temp - x+y; 

temp = (temp * alpha2M) » 6; 

temp = temp + guess2_2M; 

guess 2_2M = max (0, min (temp, 255) ) ; 

y = err4_2+err4_3; 

temp = x+y; 

temp = (temp * alpha2M)»6; 

temp = temp + guess3_2M; 

guess 3_2M = max (0, min (temp, 255) ) ; 

temp = y; 

temp = (temp * alpha2M)»5; 
temp = temp + guess 4_2M; 
guess4_2M = max (0,min (temp, 255) ) ; 



// clip value 

// guess 2_2 

// clip value 

// guess 3_2 

// clip value 

// guess 4_2 

// clip value 



int siml_l 
x 

int sim2_l = x+y; 



final0_0M+final0_lM; 
guess l_0M+guessl_lM; 
x+y; // column 1 
guess2_0M+guess2_lM; 



int sim3_l 
x 

int sim4_l = x+y; 



guess 3_0M+guess3_lM; 
x+y; 

guess4_0M+guess4_lM; 



int errl_l = (imagel_lM«2 ) - siml_l; 

int err2_l = (image2_lM«2 ) - sim2_l; 

int err3_l = (image3_lM«2) - s y im3_l; 

int err4_l = (image4_lM«2 ) - sim4_l; 

x = errl_l+errl_2; 

y = err2_l+err2_2; 

temp - x+y; 

temp = (temp * alpha3M)»6; 

temp = temp + guessl_lM; 

guess 1_1M = max ( 0, min (temp, 255 )) ; 

x = err3_l+err3_2; 

temp = x+y; 

temp = (temp * alpha3M)»6; 
temp = temp + guess2_lM; 
guess2_lM = max ( 0 , min (temp, 255 )) ; 

y = err4_l+err4_2; 

temp - x+y; 

temp = (temp * alpha3M)»6; 

temp - temp + guess 3_1M; 

guess 3__1M = max (0, min (temp, 255) ) ; 

temp = y; 

temp = (temp * alpha3M) »5; 
temp = temp + guess4_lM; 
guess4_lM = max (0, min (temp, 255) ) ; 



// column 1 



// guess 1_1 

// clip value 

// guess 2_1 

// clip value 

// guess 3_1 

// clip value 

// guess 4_1 

// clip value 



return guess 1_0M; 
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